Speech presentation system and method

ABSTRACT

A speech presentation method and apparatus which afford a dual multiple point matrix presentation of speech as patterns of points corresponding to codes representing speech phonemes and the characteristic mouth formations that produce the phonemes. Spatial patterns may be presented in either tactile or visual form, or both, from the output of a microcomputer speech analyzer that analyzes speech in real time, from a keyboard that generates phoneme- and mouth form-representing signals, or from a memory device that reproduces pre-recorded spatial patterns. The speech analyzer may be incorporated into an armband with a pair of tactile stimulator matrices to provide an unobtrusive prosthetic device for heating-handicapped individuals. A modified 16 mm projector records spatial patterns on punched film and projects the patterns onto a display to provide a visual presentation.

This application is a continuation of copending Ser. No. 435,345, filedOct. 18, 1982, which is a continuation-in-part of prior Application Ser.No. 365,772, filed Apr. 5, 1982 both abandoned.

BACKGROUND OF THE INVENTION

This invention relates generally to speech simulation systems andmethods, and more particularly to a system and method for presentingsimulated speech in a sensory perceptible form to substitute for or toaugment hearing.

Various devices and methods are known for assisting hearing-handicappedindividuals to receive speech. Sound amplifying devices, such as hearingaids, may be capable of affording a satisfactory degree of hearing tosome with a hearing impairment. Devices capable of presenting speech ina visual or tactile form have also been proposed. However, these devicestypically present speech on the basis of its frequency content, and, ingeneral, they are more useful for mimicking speech (as for speechtherapy purposes) than for interpreting speech. Consequently, they arenot very useful for assisting hearing-handicapped persons to communicatenaturally in a hearing environment.

For the deaf or those with severe hearing impairments no means isavailable that enables them to receive conveniently and accuratelyspeech with the speaker absent from view. With the speaker in view, adeaf person can speech read, i.e., lip read, what is being said, butoften without a high degree of accuracy. Also, the speaker's lips mustremain in full view to avoid loss of meaning. Improved accuracy can beprovided by having the speaker "cue" his speech using Cued Speech handforms and hand positions to convey the phonetic sounds in the message.The hand forms and hand positions convey approximately 40% of themessage and the lips convey the remaining 60%. An AutoCuer, underdevelopment by R. Orin Cornett, the originator of Cued Speech, canpresent the hand form and hand position cues for the speaker, but to usethe device the speaker's face must still be in view.

The speaker may also convert the message into a form of sign languageunderstood by the deaf person. This can present the message with theintended meaning, but not with the choice of words or expression of thespeaker. The message can also be presented by fingerspelling, i.e.,"signing" the message letter-by-letter as in Braille, or, of course, themessage can be written out and presented.

Such methods of presenting speech require the visual attention of thehearing-handicapped person which, in the long run, can visually handicapthe person because his or her visual habits become trained to favorcommunication rather than observation. Moreover, speech presentationmethods such as Cued Speech, sign language and finger-spelling are notpractical in an everyday hearing environment since they requirespecially trained speakers.

SUMMARY OF THE INVENTION

It is desirable to provide a speech simulation method and apparatuswhich avoid the foregoing disadvantages and which enablehearing-handicapped persons to receive and interpret speech convenientlyand accurately in a normal hearing environment. The inventionaccomplishes this admirably by affording a sensory perceptiblepresentation of simulated speech and other sounds that is reliable,accurate and easy to interpret, and which can substitute for or augmenthearing.

Briefly stated, in accordance with one aspect of the invention, speechis separated into constituent phonemes. Each phoneme is associated witha characteristic mouth form that produces the phoneme. The phonemes andmouth forms are encoded as distinctive spatial patterns; and the spatialpatterns are presented in a sensory perceptible form.

In accordance with another aspect, a sequence of signals representativeof speech phonemes and characteristic mouth forms that produce suchphonemes is produced. The signals are encoded as distinctive spatialpatterns, and the spatial patterns are presented in a sensoryperceptible form.

In yet another aspect, the invention provides a speech simulationapparatus that comprises means for producing a sequence of signalsrepresentative of speech, first and second arrays of tactile stimulatorsadapted to be positioned at opposed locations on a human limb to providecoordinated tactile stimulation thereto, and means responsive to thesignals for activating the tactile stimulators of each array inpredetermined patterns that correspond to signals of the sequence inorder to provide a spatially coded tactile presentation of the speech.

The invention further provides a speech simulation apparatus thatcomprises first and second means for selecting speech consonant andvowel sounds, respectively, third means for selecting mouth forms thatproduce the consonant and vowel sounds, and means responsive to thefirst, second and third means for providing signals representative ofspeech.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagrammatic view of a system embodying various aspects ofthe invention;

FIG. 2 is a diagrammatic view of matrices for presenting spatialpatterns in accordance with the invention;

FIG. 3 is a diagrammatic view of the preferred spatial pattern codes forpresenting speech phonemes and characteristic mouth forms;

FIG. 4 is a perspective view of a preferred prosthesis apparatus inaccordance with the invention for presenting tactile stimulation;

FIGS. 5A and 5B are perspective views illustrating the apparatus of FIG.4 in use;

FIG. 6 is a diagrammatic view of a preferred keyboard apparatus inaccordance with the invention;

FIG. 7 is a sectional view, partially broken away, of a portion of thekeyboard apparatus of FIG. 6;

FIG. 8 is a perspective view illustrating the keyboard apparatus of FIG.6 in use;

FIGS. 9A-9C are, respectively, a plan view, a transverse sectional viewtaken along line 9B--9B of FIG. 9A, and a longitudinal sectional viewtaken along line 9C--9C of FIG. 9A of a first tactile stimulator matrixof the apparatus of FIG. 4;

FIGS. 10A-10C are views similar to FIGS. 9A-9C, respectively, of asecond tactile stimulator matrix of the apparatus of FIG. 4;

FIGS. 11A and 11B are, respectively, an elevation view and a perspectiveview of a memory and display apparatus in accordance with the invention;

FIGS. 12A and 12B are, respectively, a perspective view, partiallybroken away, and a partial sectional view of a display apparatus thatmay be employed with the apparatus of FIGS. 11A-11B;

FIGS. 13A and 13B are, respectively, a partial sectional view and anelevation view of portions of a punch arrangement that may be employedwith the apparatus of FIGS. 11A-11B, FIG. 13B being a view along line13B--13B of FIG. 13A;

FIG. 14 is a diagrammatic view of a portion of film that may be producedby and used with the apparatus of FIGS. 11A-11B; and

FIGS. 15A and 15B are diagrammatic views illustrating areas of soundcoverage in a room, for example, such as may be afforded by differentmicrophones.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The invention is primarily intended to enable deaf or hearing-impaired(collectively referred to herein as "hearing-handicapped") individualsto communicate and function in a normal hearing environment, and it willbe described in that context. However, as will become apparent, theinvention has greater utility, and it may be employed advantageously insuch areas as speech therapy and foreign language instruction, forexample.

As will be described in detail shortly, the invention is multifaceted.In one aspect, the invention provides a speech simulation method andapparatus that afford an accurate and articulate representation ofspeech, and afford a convenient and reliable presentation of the speechin a format that is easily learned and understood. The presentation maybe presented in a tactile form as by an unobtrusive prosthetic deviceworn on the arm, thereby freeing sight to assume its normalobservational role, in a visual form, which is particularly convenientfor instructional purposes, or in both tactile and visual forms, therebyaffording a multisensory presentation. The speech presentation may be ofactual speech, produced in real-time either automatically or manually,and the presentation may be presented contemporaneously with the speechwith the innate timing and cadence of the speech. Thus, the inventioncan substitute for or augment hearing to enable hearing-handicappedindividuals to receive and comprehend speech. In addition, simulatedspeech may be recorded for later reproduction. Significantly, speech isencoded and presented in a manner that is closely related to the actualkinesthetic and physical characteristics of speech. This affords aneasily learned and readily understood code, and enables the user readilyto verify his or her comprehension of the speech by reproducing thespeech and comparing the presentations of the actual and the reproducedspeech. This aspect of the invention makes it particularly useful forspeech therapy and for learning the correct pronunciation of foreignlanguages. Having briefly indicated the more salient aspects of theinvention, it is now appropriate to describe the invention in detail.

FIG. 1 illustrates a system 20 embodying different aspects of theinvention. The figure is intended to represent the functionalrelationships among different constituent components that the system mayinclude. It does not represent a required combination of components, oreven necessarily any particular arrangement or configuration ofcomponents. As will become apparent, systems embodying the invention mayassume different forms, different configurations, and include differentcombinations of components.

As shown in FIG. 1, system 20 may include a speech analyzer 22 thatreceives as an input speech via a switch S1 that enables selection ofthe output of a general microphone 24, a directional microphone 26, or ajack 28. Jack 28 may be used for speech input to the speech analyzerfrom another type of microphone or from a sound-producing device such asa radio, television or tape recorder. Switch S1 may also have an unusedterminal 30 as for an off position.

Speech analyzer 22 preferably comprises a microcomputer speech analyzersuch as disclosed in U.S. Pat. No. 4,284,846 issued Aug. 18, 1981, toJohn Marley, the disclosure of which is incorporated by referenceherein. The Marley speech analyzer analyzes speech in real time (withapproximately a 15 msec. delay) by comparing certain computedcharacteristic parameters of the speech input signals to storedparameters to produce phoneme-representing signals. As will be describedin more detail hereinafter, in accordance with the invention each speechphoneme, i.e., a unit sound component of speech representing anutterance, is associated with a characteristic mouth form that producesthat phoneme and a sequence of signals representing a sequence of speechphonemes and their characteristic mouth forms is provided. The signalsare encoded in a predetermined manner and may be applied to a tactilestimulator 32 and/or to a visual display 34, which present the signalsas distinctive spatial multiple point grid patterns corresponding to thephonemes and their characteristic mouth forms. The speech analyzer canalso distinguish differences in pitch or speech volume, and may also beprogrammed to recognize certain environmental sounds such as a siren, adoorbell, a telephone ringer, etc., and to provide signals representingsuch sounds.

As indicated in FIG. 1, and for reasons which will be described later,the spatial patterns are preferably presented in a dual matrix formatusing a rectangular matrix for the phonemes and a square matrix for thecharacteristic mouth forms. Accordingly, tactile stimulator 32 maycomprise a rectangular array 36 and a square array 38 of tactilestimulators for providing cutaneous nerve stimulation in accordance withthe patterns. Similarly, display 34 may be formed with a rectangulardisplay area 40 and a square display area 42 for displaying the patternsas by illuminating selected portions of each area.

The output signals from speech analyzer 22 may also be applied to amemory 44 having an output adapted to drive tactile stimulator 32 anddisplay 34. The memory enables the simulated speech signals to berecorded for later replay, and enables pre-recorded signals to bereproduced as for instructional purposes.

As shown in FIG. 1, the system may also include a keyboard 50 comprisingkeyboard sections 52, 54 and 56, respectively, for manually producingphoneme- and mouth form-representing signals for driving tactilestimulator 32, display 34 and memory 44. As will be explained shortly,phonemes are preferably separated into consonant sounds and vowelsounds. Keyboard section 52 comprises a plurality of keys for producingsignals representing consonant sounds; keyboard section 54 includes aplurality of keys for producing signals representing vowel sounds; andkeyboard section 56 includes a plurality of keys for producing signalsrepresenting characteristic mouth forms. The keyboard is particularlyuseful for instructional purposes since the timing and the sequence ofsignals produced may be determined by the keyboard operator. It may alsobe used for producing pre-recorded speech using memory 44, and may beused in combination with the speech analyzer for teaching speechpronunciation as for speech therapy or foreign language instruction.

The tactile stimulator, display and memory may also be driven by anauxiliary input device 60 constructed to produce the phoneme- and mouthform-representing signals. Auxiliary input device 60 may comprise, forexample, a receiver for receiving remotely generated signals transmittedthereto as by radio waves. This would be advantageous, for example, in alecture hall or an auditorium for excluding pickup of extraneous noiseor speech signals by microphones 24 and 26. The signals produced by asingle speech analyzer positioned near a speaker's rostrum (or receivingan input from a speaker's microphone) or the signals produced by asingle keyboard operated by a speech interpreter could be modulated ontoan RF carrier that is transmitted throughout the auditorium to a numberof different receivers, each of which would demodulate the signal anddrive its associated tactile stimulator. Of course, speech outputsignals from a radio receiver, such as a walkie talkie, could also beinput directly to the speech analyzer via jack 28 to accomplish the sameeffect.

As indicated earlier, the invention contemplates systems havingdifferent combinations of the components illustrated in FIG. 1. However,prior to describing such systems, it is first appropriate to considerother significant aspects of the invention relating to the manner inwhich the encoded spatial presentations of speech are constructed.

Speech comprises sequences of phonemes corresponding to basic unit soundcomponents that are strung together to form words. The particularsequence of phonemes (sounds) and their durations determine the word orwords represented. As hereinafter described, spoken English may berepresented by 36 different phonemes. Many of these phonemes may also beused for representing sounds in other languages. However, many languagescontain sounds that are not present in English, such as, for example,the nasal vowel sounds of French and some of the more guttural sounds ofother languages, and additional phonemes may be required forrepresenting these languages. (Speech analyzer 22 has the capability ofrecognizing more than 36 phonemes, and it may be programmed to providesignals representing phonemes of other languages also.) Some languagesounds are multiple sounds and may be represented as a string of two ormore phonemes. Phonemes may also be classified as either consonantsounds or vowel sounds, and Cued Speech uses hand forms and handpositions for distinguishing between consonant and vowel sounds,respectively. However, as noted earlier, the hand forms and handpositions convey only approximately 40% of the information required tointerpret speech, and the remaining 60% must be derived by lip reading.

In accordance with the invention, speech is presented as spatiallyencoded patterns. The coding is based upon the kinestheticcharacteristics of the mouth during speech and upon the classificationof phonemes as either vowel or speech sounds. As such, it relatesclosely to active speaking and thus differs from other presentationmethods such as Braille, Morse code, and fingerspelling, which presentthe letters of the written language one-by-one, sign languages thatconvey whole word and concept meanings, and even the phonemic code inthe Gazette sections of dictionaries. It has been discovered that the 36phonemes used to represent spoken English are produced by nine differentcharacteristic mouth formations. The invention associates each phonemewith its characteristic mouth form and encodes the phoneme and mouthform as a distinctive spatial pattern. The spatial patterns arepresented in a sensory perceptible form using a dual multiple point gridmatrix format, preferably a rectangular matrix for the phonemes and asquare matrix for the mouth forms, as will be described hereinafter.

FIG. 2 illustrates a preferred rectangular matrix 70 and a preferredsquare matrix 72 for presenting the spatial patterns. As shown, therectangular matrix 70 preferably comprises a two-column by four-rowarray of "presenters", and the square matrix 72 preferably comprises athree-column by three-row array of presenters. Matrices 70 and 72 may beconstituted, for example, as arrays of tactile stimulators (such asshown at 36 and 38 in FIG. 1) or as portions of a display (such as shownat 40 and 42 in FIG. 1) that are adapted to be illuminated. One column70A of the rectangular matrix comprising four presenters 70A-1 to 70A-4is used for presenting consonant sounds (phonemes), and the other column70B comprising presenters 70B-1 to 70B-4 is used for presenting vowelsounds (phonemes). The different consonant and vowel sounds arepresented on matrix 70 by activating different combinations of thepresenters in each column. In order to make the coding as easy aspossible to learn, the presenters of column 70A may be associated withthe hand forms of Cued Speech such that each presenter or combination ofpresenters of column 70A represents a different hand form, and eachconsonant sound may be presented by activating a pattern of presenterscorresponding to the Cued Speech hand form associated with that sound.Similarly, the presenters of column 70B may be associated with the handpositions of Cued Speech, and a vowel sound may be presented byactivating presenters corresponding to the Cued Speech hand positionassociated with that vowel sound. The hand forms and hand positionswhich the different presenters (or combinations thereof) may representare indicated schematically in FIG. 2.

In column 70A, for example, each of the four presenters may individuallyrepresent the hand forms indicated schematically thereon. To illustrate,presenter 70A-1 may represent a hand form in which all four fingers areextended and the thumb is hidden from view. Presenter 70A-2 mayrepresent a hand form where the thumb and first and second fingers onlyare extended. Presenter 70A-3 may represent only the first two fingersextended, and presenter 70A-4 may represent only the index fingerextended. Other hand forms can be represented by activating differentcombinations of the presenters, as indicated by the dotted lines in thefigure. For example, a hand form (74) in which all four fingers and thethumb are extended may be represented by activating all four presentersof column 70A, as indicated. Other hand forms 75-77 may likewise berepresented, as shown.

Similarly, the presenters of column 70B may be associated with the fourhand positions indicated schematically thereon. Presenter 70B-1, forexample, may correspond to a hand position adjacent to the side of thelips, as indicted by the block dot.

To accommodate additional phonemes of other languages that are notrequired for English, the rectangular 2-by-4 matrix may be extended to a2-by-5 matrix as indicated in phantom lines in the figure. Presenter70B-5, for example, may be used for the French nasal vowels. Also,unused combinations of presenters or double patterns may be employed torepresent the additional phonemes.

The nine mouth forms (which will be described in detail shortly) thatproduce the phonemes are preferably presented on square matrix 72 byusing the presenters at the four corners and the center of the matrixindividually to represent five of the nine mouth forms (which areindicated schematically thereon). The other four mouth forms may bepresented by simultaneously activating the presenters along the twodiagonals of the matrix and along horizontal and vertical lines passingthrough the center presenter 80, as also indicated schematically in thefigure. The presenters of square matrix 72 are preferably activatedsimultaneously with the presenters of rectangular matrix 70, andtogether the two matrices present the spatial patterns representingphonemes and their characteristic mouth forms.

FIG. 3 illustrates diagrammatically preferred spatial patterns such asmay be produced by speech analyzer 22 for representing each of the 36phonemes and the nine characteristic mouth forms. As shown, there arenine groups G1-G9 of phonemes. Each group corresponds to one of the ninemouth forms, and the phonemes of each group are those that are producedby the corresponding mouth form. The square block of each groupcorresponds to square matrix 72 and illustrates the coding pattern ofthe square matrix used to represent the mouth form of that group. Therectangular blocks of each group correspond to the rectangular matrix 70and illustrate the coding patterns of the rectangular matrix used torepresent the different phonemes of the group. The letter or letter pairadjacent to each rectangular block indicates the phonemic soundsrepresented, and the darkened areas of each block indicate thepresenters of the corresponding matrix that are activated. The ninemouth forms are designated M1-M9 in FIG. 3, and they are indicatedschematically on the square matrix in FIG. 2.

The phonemes and their characteristic mouth forms, by groups, are asfollows:

GROUP G1

Sounds requiring the lips in a little round pucker (mouth form M1),represented by the center presenter 80 of matrix 72 (see FIG. 2),include "O" as in "no"; "OO" as in "book"; "OO" as in "moon" and "W" asin "was" which is the same sound with a shorter duration; and the moreblown "WH" as in "when". (The phoneme "WH" is a consonant sound, asshown by being coded in the left column (70A) of the rectangular matrixby presenter 70A-1, whereas the remaining phonemes of the group arevowel sounds and are coded in the right column of the matrix bypresenters 70B-4, 70B-3 and 70B-2, respectively.)

GROUP G2

Sounds made with an open mouth and a lowered chin (M2), encoded bypresenter 81 of matrix 72, include "H" as in "help"; "AW" as "o" in"cost"; "A" as in "patent"; "U" as in "up"; and "O" as in "dot" and"cot". ("H" is the only consonant sound of the group and is encoded by acombination of two presenters.)

GROUP G3

Sounds made with a near smile and a tense mouth (M3), encoded bypresenter 82 of matrix 72, include "I" as in "it"; "E" as in "cheese","E" as in "end"; and "Y" as in "yes".

GROUP G4

Sounds made with a tense open mouth and nasal resonance (M4), encoded bypresenter 83 of matrix 72, include "R" as in "run"; and "ER" as in"her".

GROUP G5

Throatal sounds requiring open relaxed lips (M5), encoded by presenter84 of matrix 72, include "K" as in "kick"; "G" as in "giggle"; a nearlyunheard throatal resonant "E" as in "the"; and "NG" as in "ping pong".

GROUP G6

Sounds requiring tightly closed lips (M6), encoded by presenters 80, 82and 84, include "B" as in "baby"; "P" as in "pep"; and "M" as in "mama".

GROUP G7

Sound requiring interaction of the tongue and top teeth (M7), encoded bypresenters 80, 85 and 86, include "L" as in "little"; "N" as in "none";"TH" as in "three"; "Th" as in "them"; and "D" as in "dad".

GROUP G8

Sounds requiring interaction of the lower lip and upper teeth (M8),encoded by presenters 80, 81 and 83, include "F" as in "fluffy"; and "V"as in "vivid".

GROUP G9

Sounds requiring interaction of the upper and lower teeth (M9), encodedby presenters 80, 87 and 88, include "Z" as in "zip"; "ZH" as in"measure"; "S" as in "sister"; "SH" as in "shine"; "CH" as in "cheap";"T" as in "street"; and "J" as in "judge".

Some sounds are multiple sounds that can be coded as a "gliding"sequence of two or more phonemes. These include "A as in "pay" which,pronounced one way, can be coded as M3-I to M3-E or, pronounced anotherway, as M5-E to M3-E; "I" as in "idea" can be coded as M2-A to M3-E or abit more British as M2-AW to M3-E; "X" as in "box" can be coded as M5-Kto M9-S; and "X" as in "X-ray" can be coded either as M3-E to M5-K toM9-S or as M5-E to M5-K to M9-S. Also, "E" is used with the followingsounds at word endings as a sound explosion for sound termination: "B"as in "mob"; "CH" as in "beach"; "D" as in "had"; "G" as in "dog"; "J"as in "badge"; "K" as in "stock"; "P" as in "hop"; and "T" as in "pat".For example, "mob" can be represented as the phoneme sequence "M-O-B-E".It is also possible in some cases to use a combination of M5-E and M4-Rfor "ER" rather than its own sound, especially for the "ER" ending as in"doer".

It is, of course, possible to employ a different coding scheme from thatdescribed above. Moreover, as previously indicated, the rectangularmatrix can be extended to accommodate the phonemes of other languages,and any additional mouth forms required may be presented as differentcoding patterns on the square matrix. In addition, environmental soundssuch as a siren, a doorbell, a phone ringing, or a dog barking may bepresented on the square matrix, as by activating combinations ofpresenters not passing through the center presenter 80, or as by rapidlyalternating between different combinations of presenters. It isparticularly desirable to present "emergency" sounds, e.g., a siren, toa hearing-handicapped individual to warn of danger, and this may beaccomplished with the invention. If desired, the spatial patternsemployed may be custom designed for different languages, or inaccordance with a user's preference, and may be readily adapted forbilingual or multi-lingual use.

As noted earlier, it is significant that the encoded spatial patternsrepresent not only phonemes but also their characteristic mouth forms,and as such bear a close relationship to actual speech. This isadvantageous since it allows a user to reproduce readily the "perceived"phonemes so that he or she can verify the accuracy of his or herperception. In addition, the close relationship to actual speechfacilitates learning to use the invention. Also, as is evident from theforegoing, the invention presents in a non-auditory form the same typeof information that is presented to the ear. Therefore, thehearing-handicapped individual can receive not only the speaker'smeaning, but also the speaker's choice of words and pronunciations.

FIGS. 4, 5A and 5B illustrate a preferred arrangement of the inventionfor affording tactile reception of real-time speech. Tactile stimulators36 and 38 and speech analyzer 22 may be built into the lining of anarmband 90 adapted to be worn on the forearm of the user, as illustratedin FIGS. 5A and 5B. Speech analyzer 22 and tactile stimulator 36 arepreferably formed as an integral unit, as shown, and are preferablypositioned in the armband so that they can be located on the upper sideof the forearm. Tactile stimulator 38 may be connected to the speechanalyzer and tactile stimulator 36 by a cable 92, and is preferablypositioned in the armband so that it can be located in opposingrelationship to tactile stimulator 36 on the underside of the forearm.This affords coordinated tactile stimulation of opposite sides of theforearm by the two tactile stimulators.

General microphone 24 and directional microphone 26 may be positionedwithin the armband as shown and exposed for sound reception. Themicrophones may be connected by wires 93 and 94, respectively, to switchS1 (see FIG. 5A) which may be a slide switch exposed on the armband foreasy operation. Jack 28 may also be exposed adjacent to the switch toenable input from other microphones or direct input from other speedsources. Wire 94 to directional microphone 26 is preferably long enoughto enable the directional microphone to be removed from the armband andheld extended, for example, or to enable it to be placed adjacent to thespeaker of a telephone handset. FIGS. 15A and 15B illustrate thedifferent areas of sound coverage in a room, for example, such as may beafforded by different types of microphones. General microphone 24affords complete room coverage as shown in FIG. 15A, whereas directionalmicrophone 26 allows a particular sector 95 (FIG. 15B) to be covered.Jack 28 permits other types of microphones to be used so as to providespecialized coverage of other types of areas 96 as shown in FIG. 15B. Amultipin connector 98, adapted to receive a mating connector 100 of amulti-conductor cable 101, may also be included to enable direct inputto the tactile stimulators, as from the keyboard. Preferably, the speechanalyzer, the keyboard and the memory incorporate separate sets ofdrivers for driving the tactile stimulators.

Armband 90 may be provided with suitable fasteners 102 and 104, such asVelcro™ fasteners, which extend along the sides of the armband to enableit to be securely fastened to the forearm. The armband may also includesuitable padding, and the various components such as the tactilestimulators may be suitably contoured (as hereinafter described) so asto avoid irritation to the user's forearm if it rests on the components.Preferably, the armband is worn on the forearm of the non-dominant hand,thereby freeing the dominant hand for such chores as writing, etc., andthe tactile stimulator matrices are preferably oriented so that theirvertical (in FIG. 2) axes are parallel to the longitudinal axis of thearm so as to match the user's kinesthetic sense. Also, since in use therectangular tactile stimulator 36 is upside down from the orientationillustrated in FIG. 4 (see FIG. 5A), it is formed such that the leftcolumn of tactile stimulators in FIG. 4 corresponds to the vowel side ofthe matrix and the right column corresponds to the consonant side. Thus,when the armband is in use and the matrix is positioned on the top ofthe forearm (see FIG. 5A), the vowel side of the matrix will bepositioned adjacent to the thumb side of the forearm, and the columnswill have the same relative positions as in FIG. 2. A battery (notillustrated) may also be built into the armband for providing electricalpower to the speech analyzer and to the tactile stimulators.

FIGS. 9A-C and 10A-C illustrate a preferred construction of tactilestimulator matrices 36 and 38, respectively. Each tactile stimulator,i.e., presenter, of a matrix preferably comprises a rectangularpiezoelectric crystal strip bender such as is used with devices (i.e.,an Opticon™) for presenting Braille and such as is availablecommercially under the name "Gulton Bender". As shown in FIGS. 9A-C,rectangular matrix 36 may comprise eight such benders 100 disposedwithin a housing 102, as of molded plastic, and arranged in two rows offour benders in side-by-side relationship. As best illustrated in FIGS.9B and 9C, the two rows of benders may be cantilevered from oppositesides of the housing with one row above the other. Each bender has adepending projection 104 adjacent to its free end, and the benders inthe lower row may have holes 106 therethrough to pass the projections ofthe benders of the upper row.

When a voltage (of the correct polarity) is applied to opposite sides ofa bender, its free end flexes downwardly causing projection 104 tocontact the skin, thereby providing cutaneous nerve stimulation. Thebenders may be driven by either AC or DC voltages. When driven by ACvoltages, they vibrate at the frequency of the AC voltage. The bendershave a resonant frequency that is related to their dimensions and which,in part, determines the force with which the bender projections impactthe skin. Driving the benders at different frequencies enables accentedor emphasized parts of words, or differences in pitch, to be indicated.Electrical wires (not shown) from the speech analyzer and from connector100 may be embedded within the material forming the housing and mayconnect to each bender at its fixed end. As shown in FIG. 9B, thehousing may be formed with inwardly projecting stops 108 adjacent to thefree ends of the benders to limit the amount of their downward travel,and the lengths of the projections 104 are selected to enable theprojections to contact the skin when the benders flex. Base 110 of thehousing may be contoured (FIG. 9B) so as to match the contour of theuser's forearm, and the housing may be formed with a peripheral notch112 around its exterior to enable a dust cover as of textured silk orsome other suitable material to be held over the open base of thehousing.

The square tactile stimulator matrix 38 may have a somewhat similarconstruction. However, as shown in FIGS. 10A-C, the square matrixemploys nine piezoelectric benders 120 disposed within a housing 122,which may also be of molded plastic, such that both ends of each benderare fixed within the housing, as shown in FIG. 10C. The stimulatorprojections 124 of each bender are located at the center of the benderstrip, rather than at an end as with benders 100, and when voltages areapplied to benders 120, their middle portions flex causing projections124 to contact the skin. Three benders are arranged side-by-side in atransverse direction (see FIG. 10B) and there are two rows of threebenders in a longitudinal direction as shown in FIG. 10C to provide the3-by-3 matrix. As shown in FIG. 10C, the middle benders may be supportedon inward projections 126 of the housing. The projections 126 may haveholes 128 for passing the stimulator projections 124 as best illustratedin FIGS. 10B and C. Since the benders 120 are fixed at both ends, theydo not flex as much as benders 100. Therefore, stops are not required tolimit their downward travel. The base 130 of the housing may also becontoured in a transverse direction (see FIG. 10B) to match the contourof the forearm, and the housing may have a peripheral notch 132 aboutits exterior for receiving a dust cover in the same manner as matrix 36for covering the open base 130.

Other types of tactile stimulators and other armband configurations may,of course, by employed. For example, only the tactile stimulatormatrices may be built into the armband and the speech analyzer may beconfigured as a separate unit sized to fit in a shirt or coat pocket andadapted to be connected to the tactile stimulator matrices by a cable.The configuration of FIGS. 4 and 5A-5B, however, has the advantage ofbeing entirely self-contained within the armband. If desired, thearmband may also be formed with arrays of light-emitting diodes (LED'S)positioned so as to be located on the upper side of the forearm toprovide a visual display of the spatial patterns simultaneously withtactile stimulation.

Although tactile stimulation of other parts of the body may be employed,it is preferable to use the forearm and to position the tactilestimulator matrices in opposing relationship on opposite sides of theforearm as described. This enables a coordinated stimulation of opposingsides of the forearm. By simultaneously providing tactile stimulation toopposing sides of the forearm, it has been found that the differentstimulation patterns can be distinguished more readily than if thestimulation were otherwise applied. It also permits the armband to bemore compact while affording sufficient separation between theindividual stimulators to enable stimulation by adjacent stimulators tobe easily discerned. Moreover, stimulating opposing sides of the forearmhas been found to be more comfortable to the user in that it does notproduce the same degree of irritation or annoyance that stimulating onlyone side would produce.

FIGS. 6-8 illustrate in more detail a preferred arrangement of keyboard50. As noted earlier, the keyboard may comprise three sections 52, 54and 56, each section comprising a plurality of keys for producingsignals representing, respectively, consonant sounds, vowel sounds andcharacteristic mouth forms for driving corresponding portions ofmatrices 70 and 72 (FIG. 2). As shown in FIG. 6, the keys may havethereon graphic legends which schematically indicate the hand form, handposition and mouth form signals that they produce. Also shown in FIG. 6are the spatial patterns produced by the keys.

Keyboard section 52 may comprise, as shown, twelve keys, eight of whichare used for producing signals corresponding to the eight hand forms ofFIG. 2. The bottom row of keys 140 to 143 activates presenters 70A-1 to70A-4, respectively, of rectangular matrix column 70A, as indicated bythe legends on the keys and as indicated by the spatial patterns 140' to143' in FIG. 6. Similarly, keys 144 to 147 activate differentcombinations of the presenters of column 70A as indicated by the legendson the keys and the correspondingly numbered and primed spatial patternsof FIG. 6. The four blank keys of keyboard section 52 are not requiredfor the particular coding patterns disclosed, but may be included toallow the keyboard to be adapted easily for multi-lingual speechstimulation, as by wiring the keys to produce desired patterns.

Keyboard section 54 may include five keys 150-154 corresponding,respectively, to presenters 70B-1 to 70B-5 or matrix column 70B. Thespatial pattern produced by each key, correspondingly numbered andprimed, is shown in FIG. 6. Key 154 is included to accommodate expansionof the rectangular matrix to a 2-by-5 matrix.

Keyboard section 56 may comprise five hexagonal-shaped keys 160-164 andone chevron-shaped key 165, as shown. Keys 160-164 activate presenters80-84, respectively, of square matrix 72 to produce signalscorresponding to mouth forms M1-M5, respectively. The spatial pattern onthe square matrix produced by each key, correspondingly numbered andprimed, is shown in FIG. 6. Signals corresponding to mouth forms M6 toM9 are produced by simultaneously depressing two different keys. Tofacilitate this, shallow finger depressions or indentations 166-169 maybe formed in the adjacent portions of the surfaces of keys 160-162, 164and 165, as shown. Simultaneously depressing the center key 160 and thelower left key 164 at finger depression 166 activates presenters 80, 82and 84 of the square matrix and produces a signal corresponding to mouthform M6, as indicated. Similarly, simultaneously depressing the centerkey and key 165, 161, or 162 activates appropriate combinations of thepresenters to produce signals corresponding to mouth forms M7-M9,respectively.

Keyboard section 56 may also be used for activating patterns on thesquare matrix to represent environmental sounds. For example, depressingchevron-shaped key 165 alone, or depressing different combinations ofperipheral keys 161-164 may activate three presenters of the squarematrix in a line not passing through the center presenter (80), or maycause two patterns to alternate back and forth.

FIG. 7 is a sectional view, partially broken away, of a portion ofkeyboard section 56 showing one possible construction, the figureillustrating center key 160 and peripheral key 164. As shown, each keymay have a shaft 180 depending from its underside that engages springs182 that bias the keys upwardly to their non-depressed positions. Theshafts may pass through three printed circuit boards 184 and may havedisposed thereon metallic contacts 186 adapted to engage conductorpatterns (not illustrated) on the printed circuit boards so as tocomplete electrical circuits. As shown, the upper (in the figure)contact of switch 164 and the middle contact of switch 160 engage theupper and the middle printed circuit boards, respectively, when theswitches are in their non-depressed positions. When the switches aredepressed, these contacts move out of engagement with the conductors ontheir respective printed circuit boards and break the electricalcircuits. The remaining two contacts of each switch are adapted toengage certain ones of the printed circuit boards to complete electricalcircuits when the switches are depressed. The conductor patterns on theprinted circuit boards may be connected together and to the presentersof the square matrix so that when either switch is depressed (alone) itscorresponding presenter is activated, and such that when both switchesare depressed simultaneously the appropriate combination of presentersis activated. Alternatively, keyboard section 56 may be constructed sothat each switch has a single switch contact that closes an electricalcircuit, and the electrical circuits may be configured in a logicalswitching arrangement (using digital logic circuits, for example) toprovide the desired output signals.

FIG. 8 illustrates a particularly convenient arrangement for keyboard50. As shown, keyboard sections 52 and 54 may be configured to besecured to a leg (for example, the right leg) of the user as by a strapand buckle arrangement 190, and keyboard section 56 may be formed to besecured to the thumb of the left hand of the operator, as by anotherstrap and buckle arrangement 192, as shown. This arrangement allowskeyboard section 56 to be operated with the fingers of the left hand,while simultaneously allowing keyboard sections 52 and 54 to be operatedwith the fingers and the thumb, respectively, of the right hand.Keyboard section 54 may be hinged as shown at 194 to keyboard section 52and movably supported on a member 195 attached to the hinge to enablethe keyboard sections to be positioned at an angle and locationconvenient for the operator. The keyboard sections may also have acontoured base, as shown at 196 for keyboard section 54, to conform tothe user's leg. Keyboard section 56 may be provided with a contouredextension 198, as of plastic or rubber, adapted to fit in the palm ofthe user's hand and to extend around the wrist to the back of the handto prevent keyboard section 56 from rotating about the user's thumb. Acable 200 may be used for electrical connection between keyboard section56 and keyboard sections 52 and 54 as shown, and another cable 202 maybe used for the output signals from the three keyboard sections. Cable202 may correspond to cable 101 (FIG. 4), for example.

As noted earlier, the keyboard may provide output signals to the tactilestimulator, to the display or to the memory by direct electricalconnection thereto, or the signals may be modulated onto radio waves andtransmitted to an auxiliary input device connected to these components,or to a radio receiver connected to jack 28 of the speech analyzer. Thisarrangement, as previously indicated, is particularly advantageous foruse in a lecture hall or the like where one speech interpreter operatinga keyboard can serve a number of different individuals.

Memory 44 and display 34 of system 20 (FIG. 1) may assume a number ofdifferent forms. For example, the memory might comprise a tape recorderfor recording the signals representing the spatial patterns on magnetictape and may be configured to provide an output to drive tactilestimultor 32 and display 34. By using a dual track tape recorder, anaudio output of the speech may also be provided simultaneously with thespatial patterns, which would be particularly convenient for speechtherapy or for foreign language instruction. Display 34 may assumedifferent forms such as, for example, a light panel configured topresent the spatial patterns as illuminated areas. FIGS. 11A-B, 12A-Band 13A-B illustrate an apparatus that combines both a memory and adisplay.

As shown in FIG. 11A, a memory 44 may comprise a modified 16 mmprojector 240 configured to preserve the spatial patterns on punchedtape or film 242 and to display the spatial patterns by projecting themonto a display panel 244. (FIG. 11B shows the projector and displaytogether for easy transportability.) To punch the film, projector 240may include a punch 246, one form of which is illustrated in FIG. 13A,adapted to be driven by either the speech analyzer or the keyboard andto be operated at the 24 frame per second rate of the projector so as tobe synchronous with the film advance mechanism. Punch 246, which islocated ahead of the shutter, may be a mechanical punch comprising aplurality of hole punches arranged in the 2-by-4 (or 2-by-5) and 3-by-3matrix configurations that impact upon a backing plate 248 (shown inFIG. 13B) having a corresponding arrangement of holes 250 therein. Thepunch may also be a laser diode punch (see FIG. 13A) comprising arraysof laser diodes 252 and associated focusing lenses 254 arranged inaccordance with the matrix configurations. The projector may alsoinclude an exhaust or collection system (not illustrated) behind backingplate 248 to collect the punched debris from the film, and may include aguide 258 for indicating the correct distance of display 244 form theprojector for proper positioning of the projected patterns on thedisplay.

FIG. 14 illustrates a strip 260 of punched film such as may be producedby the invention, the film being illustrated as it would appear ifviewed from the rear of projection lamp 262 through the backing plateand projection lens 264 of projector 240 (see FIG. 11A). (For ease ofillustration, the spatial patterns on the film are illustrated asblackened areas, whereas on the actual film the background wouldpreferably be black and the spatial patterns would appear as clearareas.) The figure illustrates the sequence of spatial patternscorresponding to the constituent phonemes of one pronunciation of theword "patents", the film being read from the bottom to the top of thefigure corresponding to the direction in which it is advanced throughprojector 240. The phoneme spatial patterns appear on successive framesF1-F15, and the phoneme represented by each pattern is indicated on theleft side of the figure.

The film presents not only the phonemes in their proper sequence, butalso indicates their timing, i.e., duration. As shown, frame F1 containsthe spatial pattern for the phoneme "P", which (for this example) has aduration of only one frame, whereas frames F2-F6 contain the spatialpattern for the phoneme "A", indicating that its duration is four timesthat of phoneme "P". Frames F7 and F8 contain the spatial patterns forthe phonemes "T" and "E", respectively, each having a duration of onlyone frame. Frames F9-F11 contain the spatial pattern for the phoneme"N", which has a duration of three frames; frames F12 and F13 containthe spatial patterns for the phonemes "T" and "E", respectively, andframes F14 and F15 contain the spatial pattern for the phoneme "S".

The phoneme sequence illustrated in FIG. 14 corresponds to thepronunciation "pa-ten-tes". "Patents" may also be pronounced as"pa-ten-z", in which case the spatial patterns in frames F12-F15 (andperhaps additional frames depending upon the duration of the ending)would be the spatial pattern representing the phoneme "Z". Thisillustrates a significant aspect of the invention. Namely, the inventionpresents the same information that would be presented to the ear, i.e.,the actual phonemic content of speech, which may vary depending uponpronunciation, as well as the timing and cadence of the actual speech.

The punched film may be projected and viewed as it is produced withapproximately a two-frame (1/12 second) delay. As the film is punched,the speech may also be recorded on a sound track 270 of the film, andmay be reproduced simultaneously with the spatial patterns so as toafford a multi-sensory presentation. The punched film also has theadvantage that it can be read both visually (without projecting it) andtactilely (much like Braille) by running a finger across the film tosense the punched patterns. Thus, deaf-blind individuals can easilyidentify a particular film or portion thereof that they wish to review.Furthermore, projector 240, which normally runs at the standard 24frames per second rate, can be slowed down or advanced in a stop-framemode, as for instruction or demonstration purposes, without distortingeither the visual presentation or a tactile presentation derivedtherefrom.

FIGS. 12A and 12B illustrate in more detail a preferred form of display244 which may be built into the end cover of the projector 240 for easytransportation, as illustrated in FIG. 11B. Preferably, display 244comprises a ground glass projection surface 270 onto which the spatialpatterns of the punched film may be projected, as shown in FIG. 11A. Abacking plate 272 having hole patterns corresponding to the rectangularand square matrices may be positioned behind the ground glass projectionsurface, and reflectors 272 having light sources 276 therein may belocated within the holes of the backing plate, as illustrated in FIG.12B. Preferably, the backing plate and the reflectors are selected so asto afford a substantially constant surface reflectivity to the groundglass projection surface so that when viewed from the front the surfaceappears uniform. Photocells or other light responsive devices 278 may bepositioned at the rear surface of the projection surface within eachreflector and slightly offset from the center line of the reflector, asshown in FIG. 12B.

Display 244 may be used in a number of different ways. The display maybe used only as a projection screen for viewing the spatial patternsprojected thereon by projector 240. Display 244 may also be used withoutprojector 240 as a rear projection type display for displaying thespatial patterns by driving lamps 276 directly from the speech analyzer,the keyboard, or an auxiliary device, e.g., a tape recorder. Photocells278 respond to the patterns of light projected onto the ground glasssurface by the projector or produced by lamps 276, and produce outputsthat may be used for driving the tactile stimulators to provide tactilestimulation simultaneously with the visual display. Display 244 can alsobe used conveniently for speech therapy or instructional purposes sinceit enables a student to compare his or her speech attempts to "model"speech, as by simultaneously presenting the model speech and thestudent's speech using different colors.

While preferred embodiments of the invention have been shown anddescribed, it will be apparent to those skilled in the art that changescan be made in these embodiments without departing from the principlesand spirit of the invention, the scope of which is defined in theappended claims.

The invention claimed is:
 1. A method of presenting speech informationcomprising separating speech into a sequence of phonemes representativeof the speech; transforming each phoneme into first and second gridpatterns of points, the first pattern being a code which identifies thephoneme sound and the second pattern being another code which identifiesa mouth form that produces the phoneme sound; and presenting the firstand second patterns by activating presenters arranged in a multiplepoint grid matrix in accordance with said patterns of points.
 2. Themethod of claim 1, wherein said separating comprises identifyingsuccessive phonemes and their durations, and wherein said presentingcomprises successively presenting the first and second patterns with thesame durations as the phonemes to which they correspond.
 3. The methodof claim 2, wherein said presenting comprises presenting the patternssubstantially contemporaneously with the speech.
 4. The method of claim1, wherein said presenting comprises presenting the first patterns on afirst multiple point grid matrix array of presenters and presenting thesecond patterns on a second multiple point grid matrix array ofpresenters.
 5. The method of claim 4, wherein the presenters of thefirst array comprise first and second groups of presenters, the firstgroup being employed for representing consonant sounds and the secondgroup being employed for representing vowel sounds.
 6. A speechpresentation apparatus comprising first keyboard means for selectingspeech phonemes corresponding to consonant sounds, second keyboard meansfor selecting speech phonemes corresponding to vowel sounds, thirdkeyboard means for selecting mouth forms that actually produce saidconsonant and said vowel sounds, each keyboard means having a pluralityof keys for enabling the selections, and means responsive to the first,second and third keyboard means for providing signals representative ofsaid phonemes and mouth forms.
 7. The method of claim 1, wherein saidtransforming comprises associating each phoneme sound with one of apreselected group of mouth forms.
 8. The method of claim 1, wherein saidpresenting comprises presenting the patterns as tactile stimuli.
 9. Themethod of claim 1, wherein said presenting comprises presenting thepatterns as visual stimuli.
 10. The method of claim 1, wherein saidpresenting comprises presenting the patterns simultaneously as tactileand visual stimuli.
 11. The method of claim 1, wherein said presentingcomprises presenting the first and second patterns at differentlocations.
 12. A method of presenting speech information comprisingproducing a sequence of signals representative of speech phonemes;transforming the signals into a sequence of first and second gridpatterns of points, the first pattern being a code which identifies thephoneme sound and the second pattern being another code which identifiesa mouth form that produces the phoneme sound; and presenting the firstand second patterns by activating presenters arranged in a multiplepoint grid matrix in accordance with said patterns of points.
 13. Themethod of claim 12, wherein said transforming comprises analyzing thespeech in real time to identify successive phonemes, and forming acorresponding sequence of said first and second patterns, and whereinsaid presenting comprises presenting the patterns substantiallycontemporaneously with the speech.
 14. The method of claim 12, whereinsaid producing comprises generating said sequence of signals using akeyboard.
 15. The method of claim 14 wherein said producing comprisesproducing with a first plurality of keys first signals representingconsonant sounds, producing with a second plurality of keys secondsignals representing vowel sounds, and producing with a third pluralityof keys third signals representing mouth forms.
 16. The method of claim12, wherein said producing comprises producing a preselected sequence ofsignals from a storage medium.
 17. The method of claim 12, wherein saidpresenting comprises presenting the first and second patterns on firstand second arrays of presenters.
 18. The method of claim 17, whereinsaid first and second arrays comprise first and second tactilestimulator matrices for presenting the patterns as tactile stimuli. 19.The method of claim 18, wherein said presenting comprises positioningthe tactile stimulator matrices at first and second opposed areas of ahuman limb and presenting the tactile stimuli at said first and secondareas.
 20. The method of claim 19, wherein said presenting comprisespresenting the patterns as visual stimuli simultaneously with saidtactile stimuli.
 21. The method of claim 12 further comprising producingsignals representative of preselected environmental sounds, and whereinsaid presenting comprises presenting third patterns representative ofthe preselected sounds.
 22. Apparatus for presenting speech informationcomprising means for separating speech into a sequence of phonemesrepresentative of the speech; means for transforming each phoneme intofirst and second grid patterns of points, the first pattern being a codewhich identifies the phoneme sound and the second pattern being anothercode which identifies a mouth form that produces the phoneme sound; andmeans for presenting the first and second patterns, the presenting meansincluding multiple point grid matrix arrays of presenters, and means foractivating the presenters in accordance with said patterns of points.23. The apparatus of claim 22, wherein said separating means comprisesmeans for analyzing the speech in real time to identify successivephonemes and their durations, and wherein the presenting means comprisesmeans for presenting corresponding patterns with the same durations andsubstantially contemporaneously with the speech.
 24. The apparatus ofclaim 23, wherein the analyzing means comprises means for classifyingeach phoneme as a consonant sound or as a vowel sound.
 25. The apparatusof claim 24, wherein the presenting means comprises a first array ofpresenters for presenting said second patterns and a second array ofpresenters for presenting said second patterns.
 26. The apparatus ofclaim 25, wherein the first array of presenters comprises a rectangularmatrix having two columns of presenters, one column being used forpresenting consonant sounds and the other column being used forpresenting vowel sounds, and wherein the second array comprises a squarematrix of presenters for the mouth forms.
 27. The apparatus of claim 25,wherein the first and second arrays comprise tactile stimulators forpresenting the patterns as patterns of cutaneous nerve stimulation. 28.The apparatus of claim 27, wherein the first and second arrays oftactile stimulators are built into the lining of an armband adapted tobe worn on the forearm of a user, the tactile stimulator arrays beingarranged so as to be positioned in opposing relationship on oppositesides of the forearm to provide coordinated tactile stimulation thereto.29. The apparatus of claim 22, wherein the separating means and thetransforming means comprise a microcomputer speech analyzer having inputmeans for speech signals and having means for selecting said speechsignals from a plurality of speech sources.
 30. The apparatus of claim29, wherein said plurality of speech sources comprises a plurality ofmicrophones, each having a different area of coverage.
 31. The apparatusof claim 22 further comprising memory means for storing said codesidentifying phoneme sounds and mouth forms, and wherein the presentingmeans is responsive to the memory means for presenting said first andsecond patterns.
 32. The apparatus of claim 31, wherein the memory meanscomprises film upon which spatial patterns representing the phonemes andmouth forms are recorded as punched patterns, and wherein the presentingmeans comprises means for projecting the punched patterns to produce avisual presentation thereof.
 33. Apparatus for presenting speechinformation comprising first means for producing a sequence of signalsrepresentative of speech phonemes; second means for transforming thesignals into a sequence of first and second grid patterns of points, thefirst pattern being a code which identifies the phoneme sound and thesecond pattern being another code which identifies a mouth form thatproduces the phoneme sound; and means for presenting the first andsecond patterns, the presenting means including multiple point gridmatrix arrays of presenters, and means for activating the presenters inaccordance with said patterns of points.
 34. The apparatus of claim 33,wherein the producing means comprises means for separating speech intoits constituent phonemes, and means for associating with each phoneme acharacteristic mouth form from a preselected group of mouth forms. 35.The apparatus of claim 34, wherin said separating means comprises meansfor analyzing the speech in real time to identify successive phonemesand their durations, and wherein said presenting means comprises meansfor successively presenting corresponding patterns having the samedurations.
 36. The apparatus of claim 33, wherein producing meanscomprises a keyboard having a first plurality of keys for producingfirst signals representing consonant sounds, a second plurality of keysfor producing second signals representing vowel sounds, and a thirdplurality of keys for producing third signals representing the mouthforms.
 37. The apparatus of claim 33, wherein said producing meanscomprises a memory that reproduces a stored sequence of said signals.38. The apparatus of claim 33 further comprising means for transmittingsaid signals to a plurality of such presenting means.
 39. The apparatusof claim 33, wherein the presenting means comprises first and secondarrays of presenters, the first array presenting the first patternscorresponding to and the second array presenting the second patterns.40. The apparatus of claim 39, wherein said presenting means comprisesfirst and second tactile stimulator matrices.
 41. The apparatus of claim39, wherein the presenting means comprises means for presenting thepatterns as illuminated areas on a display.
 42. The apparatus of claim41, wherein the presenting means comprises means for projecting thepatterns onto the display.
 43. The apparatus of claim 42, wherein thedisplay includes means for illuminating preselected portions thereof soas to display patterns that can be compared with a spatial patternprojected onto the display by the projecting means.
 44. The apparatus ofclaim 43, wherein the display comprises a ground glass plate having afront surface for displaying the patterns projected thereon and having arear surface at which a plurality of light sources arranged inaccordance with the first and second arrays is positioned.
 45. Theapparatus of claim 41, wherein the display comprises means for producingelectrical signals representative of the patterns displayed thereon fordriving another presenting means.
 46. Apparatus for presenting speechinformation comprising means for producing a sequence of signalsrepresentative of speech; first and second multiple point grid matrixarrays of tactile stimulators adapted to be positioned at spacedlocations on a human limb for providing coordinated tactile stimulationthereto; and means responsive to the signals for simultaneouslyactivating the tactile stimulators of each array in respective first andsecond sequences of patterns that correspond to signals of the sequence,the first pattern being a code which represents a speech sound and thesecond pattern being another code which represents a mouth shape thatproduces the speech sound, in order to provide a coded tactilepresentation of the speech.
 47. The apparatus of claim 46, wherein thefirst and second arrays are adapted to be positioned in opposingrelationship on opposite sides of the limb.
 48. The apparatus of claim47, wherein the limb comprises a forearm, and the first and secondarrays are mounted in an armband adapted to be worn on the forearm. 49.The apparatus of claim 48, wherein the producing means comprises aspeech analyzer mounted in the armband for producing signalsrepresentative of speech phonemes and characteristic mouth forms thatactually produce such phonemes.
 50. The apparatus of claim 49, whereinthe tactile stimulators comprise piezoelectric strip benders havingprojections adapted to contact the forearm.
 51. The apparatus of claim48, wherein the tactile stimulators of each array are spaced from oneanother sufficiently to enable stimulation of adjacent areas of the limbto be readily distinguished.
 52. The apparatus of claim 6, wherein thefirst and second keyboard means comprise first and second keyboardsformed to be mounted on a leg of a user and to be operated with one handof the user, and wherein the third keyboard means comprises a keyboardadapted to be mounted on the thumb of the opposite hand of the user andto be operated by the fingers thereof.
 53. The apparatus of claim 52,wherein the first and second keyboards are movably related so as toenable their relative positions with respect to one another to bevaried.
 54. The apparatus of claim 27 further comprising means forvibrating said tactile stimulators at different frequencies to presentdifferences in speech pitch.